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EXECUTIVE  SUMMARY 

This  report  details  efforts  to  date  to  advance  the  state  of  the 
art  of  Quality  Assurance/Quality  Control  (QA/QC)  in  waste  site 
characterization  and  remedial  action.  The  report  is  organized  into 
three  sections.  The  first  section  provides  recommendations  on  the 
unification  of  the  two  widely  used  and  accepted  QA/QC  programs:  the 
U.S  Environmental  Protection  Agency  Contract  Laboratory  Program  (USEPA 
CLP)  and  the  U.S.  Army  Toxic  and  Hazardous  Materials  Agency 
Installation  Restoration  Quality  Assurance  (USATHAMA  IR  QA)  plan.  The 
second  section  compares  the  two  plans  in  detail  with  the  Guidelines 
given  by  the  USEPA.  The  third  section  announces  the  formation  of  a 
Task  force  on  Quality  Assurance/Quality  Control  and  reports  on  the 
findings  of  a  Working  Group  convened  to  address  these  issues. 
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UNIFICATION  OF  THE  USATHAMA  IR  QA  PLAN  WITH  THE  USEPA 
CONTRACT  LABORATORY  PROGRAM 


There  currently  exists  a  widespread  agreement  on  the  need  for 
remedial  action  at  past  waste  disposal  sites.  The  approach  usually 
taken  is  to  study  records  pertaining  to  the  site  (preliminary 
investigation) ,  to  follow  the  investigation  with  a  survey  of 
contamination,  and  then  to  decide  on  a  remedial  action  plan.  Privately 
used  sites  are  regulated  under  the  Super fund  Amendment  and 
Reauthorization  Act  (SARA)  by  the  USEPA.  Sites  used  by  government 
agencies,  such  as  DOD  and  DOE,  while  regulated  under  SARA,  are  not 
generally  cleaned  up  using  SARA  funds.  An  important  aspect  of  the 
entire  process  is  the  analysis  of  large  numbers  of  samples.  Because  of 
the  increased  emphasis  on  analytical  methodologies,  and  the  associated 
cost,  it  is  crucial  to  ensure  that  the  data  produced  be  of  acceptable 
quality.  Therefore,  strict  Quality  Assurance  (QA)  measures  must  be 
applied.  Several  different  approaches  to  the  various  aspects  of  QA 
have  been  developed  over  the  last  decade,  with  perhaps  the  best  known 
being  the  approach  used  by  the  USEPA  under  the  Contract  Laboratory 
Program  (CLP).  In  addition,  the  U.S.  Army  Toxic  and  Hazardous 
Materials  Agency  (USATHAMA)  has  developed  a  QA  plan  to  serve  the  needs 
of  the  Army  Installation  Restoration  program.  Both  systems  have  been 
in  existence  for  several  years,  and  substantial  experience  has  been 
gained.  Because  of  the  obvious  similarity  in  the  objectives  of  the  two 
systems,  this  work  was  performed  in  an  attempt  to  draw  the  two  plans 
together.  This  was  done  in  the  following  manner.  A  workshop  was  held 
to  bring  all  interested  parties  together  and  discuss  differences  and 
similarities.  Then,  a  detailed  comparison  was  made  of  the  two  plans. 
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Finally,  this  report  was  written  to  document  the  results  of  these 
efforts . 

It  must  be  noted  that  certain  differences  exist  in  the  two 
programs  which  result  from  philosophical  and  logistical  considerations 
beyond  the  issue  of  QA.  Virtually  all  data  generated  in  the  CLP  may 
eventually  be  called  into  court  as  evidence  for  prosecution  or  cost 
recovery  actions.  Therefore,  it  is  necessary  that  the  data  be  of 
courtroom  quality.  In  the  case  of  the  IR  plan,  use  of  the  data  is 
generally  restricted  to  direction  of  the  remedial  action  phase.  While 
not  removing  the  requirement  for  high  quality  data,  this  end  use  does 
not  mandate  the  degree  of  documentation  used  in  the  CLP.  More 
important  in  the  IR  plan  is  streamlining  of  the  data  flow,  and  rapid 
identification  of  QA  problems.  In  addition,  the  CLP  has  a  relatively 
large  number  of  participating  laboratories  compared  to  the  IR  system. 
This,  combined  with  the  end  use  difference,  results  in  a  need  for  more 
rigid  standardization  of  the  entire  QA  process.  Inter laboratory 
comparability  becomes  much  more  crucial  to  the  CLP  than  to  the  IR 
program.  A  related  logistical  difference  is  the  fact  that  in  the  CLP, 
samples  are  collected  and  sent  to  a  sample  management  facility,  either 
central  or  regional.  The  samples  are  then  distributed  to  the 
analytical  facilities.  In  the  USATHAMA  case,  samples  are  collected  by 


e 
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the  prime  contractor  for  the  remedial  action  and  either  analyzed  in 
house  or  sent  to  a  subcontracting  analytical  facility.  The  CLP  can 
therefore  do  a  more  effective  job  of  providing  blind  QA  samples 
(spikes,  splits,  or  blanks)  than  can  the  IR  plan.  Taken  together, 
these  differences  result  in  a  marked  reliance  on  external  QA  in  the 
CLP,  and  a  corresponding  reliance  on  internal  QA  in  the  IR  plan.  Given 
this  difference,  it  would  appear  that  the  two  systems  are  not  mutually 
exclusive,  and  that  reconciliation  of  the  programs  might  result  in  an 
even  stronger  unified  plan. 

The  first  issue  to  be  resolved  is  the  method  of  assuring  initial 
laboratory  proficiency.  In  the  CLP,  this  is  done  by  analysis  of  a 
performance  evaluation  sample.  In  the  IR  plan,  the  laboratory  performs 
a  certification  study,  which  is  used  to  establish  the  QA  paramaters  for 
the  method.  While  not  specifically  required  by  the  CLP,  some  type  of 
initial  certification  must  certainly  be  done  by  the  laboratory  in  order 
to  gain  familiarity  with  the  method  prior  to  running  the  PE  sample.  It 
would  seem  prudent  to  establish  guidelines  for  the  certification 
process  which  allow  the  laboratory  to  prove  competence  in  the  method. 
The  certification  procedure  used  by  USATHAMA  should  be  recommended  or 
even  required  by  the  CLP. 

The  use  of  performance  evaluation  samples  has  advantages  and 
disadvantages.  PE  samples  can  rarely  be  provided  which  are  truly 
blind.  There  is  no  assurance  that  the  successful  analysis  of  a  PE 
sample  reflects  everyday  laboratory  performance.  Furthermore,  the 
analysis  of  PE  samples  is  restricted  to  a  relatively  low  frequency 
(quarterly)  so  that  if  problems  are  identified,  large  gaps  exist  in 
which  laboratory  performance  is  in  question.  Finally,  the  time 
required  to  analyze  the  PE  data  further  increases  the  lag  time.  On  the 
other  hand,  the  PE  sample  is  the  only  truly  external  check  on 
laboratory  performance,  and  the  only  means  by  which  laboratories  can  be 
compared  and  rated.  Therefore,  It  is  recommended  that  USATHAMA  adopt 
the  performance  evaluation  system  used  by  the  CLP,  and  that  efforts  be 
made  to  rapidly  evaluate  that  data  and  report  problems  to  the 
laboratory. 

Several  differences  exist  between  the  two  plans  with  respect  to 
sampling  and  analysis.  The  only  fundamental  difference  is  that  the  CLP 
requires  the  analysis  of  all  samples  for  compounds  on  the  Hazardous 
Substances  List  (HSL)  while  the  Army  has  contamination  from  military- 
specific  compounds  which  do  not  appear  on  the  HSL.  When  the  IR  plan  is 
used  for  analysis  of  HSL  compounds,  the  CLP  methodology  is  followed. 
Therefore,  the  IR  plan  is  equivalent  from  an  analytical  standpoint  to 
the  CLP,  but  includes  in  addition  the  QA  required  for  non-CLP  methods. 
Other  differences  include  container  cleaning  procedures  and  holding 
times.  Differences  of  this  type  can  be  handled  experimentally,  by 
performing  an  equivalency  test.  The  IR  plan  does  not  require  chain-of- 
custody  procedures  to  be  followed,  unless  the  data  is  to  be  used  in 
litigation.  When  used,  these  procedures  are  functionally  identical  to 
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those  used  in  the  CLP  plan.  Chain-of -custody  procedures  are  a 
necessary  part  of  good  laboratory  practice,  and  should  always  be  used. 

It  is  recommended  that  USATHAMA  require  CLP  chain-of -custody  procedures 
be  followed  for  all  samples. 

In  terms  of  data  management  and  communication,  USATHAMA  has 
developed  a  sophisticated  computer-based  system.  All  data  is  entered 
by  the  analyst  into  a  personal  computer,  checked  for  completeness,  and 
transferred  to  a  mainframe.  The  laboratory  is  required  to  submit  all 
raw  data  at  the  end  of  the  contract.  In  the  CLP,  all  data  generated 
pertaining  to  a  particular  sample  is  submitted  with  the  results  from 
that  sample  in  a  data  package.  This  is  clearly  an  example  of 
differences  resulting  from  the  end-use  situation  mentioned  previously. 
The  USATHAMA  system  is  far  more  workable  from  the  standpoint  of 
remedial  action  decision  making,  but  the  CLP  system  is  required  for 
litigation.  However,  it  must  be  pointed  out  that  the  data  in  either 
case  is  available,  and  that  nothing  has  been  lost.  Therefore,  it 
should  be  possible  for  the  IR  software  to  produce  a  CLP  data  package  on 
demand.  If  this  can  be  done,  then  there  is  no  practical  reason  to 
change  either  plan.  It  is  recommended  that  USATHAMA  demonstrate  the 
ability  to  produce  a  CLP  data  package. 

The  software  package  used  in  the  IR  plan  has  additional  features 
which  are  quite  desirable  from  a  QA  standpoint,  including  the  ability 
to  generate  QC  charts.  While  it  is  implied  in  the  CLP  that  QC  charts 
should  be  kept,  no  formal  requirement  exists  and  no  standardized 
approach  is  provided.  QC  charting  has  several  advantages:  rapid 
identification  of  out-of -control  situations,  assurance  that  performance 
is  consistent  on  a  day-to-day  basis,  and  documentation  that  the 
laboratory  is  performing  well  on  each  and  every  sample.  Thus,  QC 
charting  can  serve  as  an  adjunct  to  the  PE  system,  and  alleviate  the 
drawbacks  of  PE  samples.  The  question  is:  what  should  be  charted? 
Since  the  surrogates  and  internal  standards  used  in  the  CLP  are  present 
in  every  sample,  it  seems  logical  to  require  that  the  surrogate 
recoveries  and  internal  standard  areas  be  control  charted.  It  is 
recommended  that  the  CLP  use  the  USATHAMA  software  package  and  require 
control  charts  for  surrogate  recoveries  and  internal  standard  areas. 
It  is  further  recommended  that  USATHAMA  provide  USEPA  with  the  software 
and  documentation. 

A  difference  also  exists  in  the  area  of  matrix  spiking.  The  CLP 
requires  a  matrix  spike  and  matrix  spike  duplicate  to  be  run  for  each 
matrix.  The  IR  plan  uses  a  standard  matrix.  The  CLP  matrix  spike  does 
provide  additional  information  on  the  performance  of  the  methods  with 
respect  to  individual  matrices.  However,  it  can  be  difficult  to 
determine  when  one  matrix  differs  from  the  previous  one.  On  the  other 
hand,  the  IR  method  provides  a  historical  record  of  the  performance  of 
the  method  with  time.  Given  that  the  surrogates  are  present  in  every 
sample  and  can  be  considered  matrix  spikes,  the  issue  seems  to  be 
whether  any  additional  information  can  be  obtained  from  sample  matrix 
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spikes.  Furthermore,  the  issue  of  interlaboratory  comparability  -  so 
important  to  the  CLP  program  -  would  be  better  served  by  the  use  of  a 
standard  matrix  than  by  use  of  sample  matrices.  It  is  recommended  that 
the  CLP  drop  the  requirement  for  sample  matrix  spikes  and  matrix  spike 
duplicates  and  adopt  the  standard  matrix  approach  used  by  USATHAMA. 


One  of  the  major  problems  faced  by  analytical  laboratories  doing 
work  in  the  remedial  action  area  is  the  audit.  Each  contracting  agency 
has  its  own  style  of  auditing,  and  preparation  for  the  audit  depends  on 
the  needs  and  requirements  of  the  auditing  agency.  In  both  the  CLP  and 
the  IR  programs,  the  audit  is  used  as  a  tool  to  improve  the  performance 
of  the  laboratory.  Because  of  all  of  the  differences  listed 
previously,  the  audits  take  on  a  different  flavor  depending  on  which 
agency  is  auditing.  However,  if  the  modifications  recommended  in  this 
document  can  all  be  made,  the  audit  could  be  performed  by  either 
USATHAMA  or  CLP  personnel  and  would  suit  the  needs  of  both  programs. 
This  would  result  in  substantial  savings  to  the  agencies  involved  and 
would  be  very  convenient  for  the  laboratories. 

In  summary,  the  recommendations  made  here  are  the  result  of  an 
objective  comparison  of  the  CLP  and  the  USATHAMA  IR  QA  plan.  These 
recommendations  are  made  with  the  goal  of  improving  quality  assurance 
and  quality  control  in  environmental  measurements  related  to  waste  site 
characterization  and  remedial  action.  An  additional  goal  is  the 
reduction  of  the  cost  associated  with  QA.  Two  approaches  are  feasible 
in  this  regard.  The  most  easily  adopted  from  the  philosophical 
viewpoint  is  the  declaration  of  equivalency  of  the  two  plans.  To  this 
end,  a  detailed  comparison  follows  of  the  two  plans  with  the  general 
guidelines  set  forth  in  the  USEPA  sixteen  point  QA  project  plan.  While 
this  would  be  expedient,  the  separate-but-equal  approach  is  far  less 
desirable  than  the  approach  of  combining  the  best  of  both.  To  that 
end,  continuing  communication  between  the  principal  agencies  and  a 
willingness  to  cooperate  on  these  issues  is  mandatory.  It  is 
recommended  that  the  USEPA  grant  equivalency  to  the  USATHAMA  IR  QA 
plan,  but  at  the  same  time  strive  for  unification. 
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certification  Performance  Data 
Package  and  Project  QA  Plan 


Tabulation  of  found  vs.  target  Sample  Traffic  Report 

concentration  Sample  Data  Summary  Package 
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(a)  One  *  standard  matrix  method  blank 

(b)  One  standard  matrix  spike  -  1  CRL  Spiked  sample  and  duplicate  sample  - 

at  least  one  for  each  group  of  samples 
of  a  similar  matrix  and  concentration 
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APPENDIX  G 

SAMPLE  CONTAINER  CLEANING  PROCEDURES 


To  ensure  the  integrity  of  aqueous  and  solid  samples,  steps  must  be  taken 
to  minimize  contamination  frcm  the  containers  in  which  they  are  stored.  If 
the  analyte (s)  to  be  determined  are  organic  in  nature,  the  container  should  be 
made  of  amber  glass.  If  the  analyte (s)  are  inorganic,  the  container  should  be 
polyethylene.  Uhen  both  organic  and  inorganic  substances  are  e:rpected  to  be 
present,  separate  samples  should  be  taken.  New  sample  bottles  must  be  cleaned 
according  to  the  procedure  presented  below;  reuse  of  sampie  containers  is 
expressly  prohibited.  Carmercially  cleaned  containers  may  be  utilized  if 
cleaning  procedures  comply  with  those  provided  in  this  appendix  and  prior 
USATKAMA  approval  is  obtained.  The  procedures  for  cleaning  the  glass  and 
polyethylene  containers  ana  their  caps  are  as  follows: 

•  Polyethylene  Bottles  and  Polyethylene  Caps 

(1)  Rinse  bottles  and  lids  with  5%  sodium  hydroxide. 

(2)  Rinse  with  deionized  water. 

(3)  Rinse  with  5%  Ultrex  (or  equivalent)  nitric  acid  in  deionized 
water . 

(4)  Rinse  with  deionized  water. 

(5)  Drain  and  air  dry'. 

•  Amber-Glass  Bottles  or  40-ml  Vials 

(1)  Scrub  and  wash  bottles  in  detergent. 

(2)  Rinse  with  copious  amounts  of  distilled  water. 

(3)  Rinse  with  acetone. 

(4)  Rinse  with  methylene  chloride  (Nanograde  or  equivalent) . 

(5)  Rinse  with  hexane  (Nanograde  or  equivalent) . 

(6)  Air  dry. 

(7)  Heat  to  2C0°C. 

(8)  Allow  to  cool. 

(9)  Cap  with  clean  caps  with  Teflon  liners. 
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(1)  Remove  paper  liners  iron  caps. 

(2)  Wash  with  detergent. 

(3)  Rinse  with  distilled  water. 

(4)  Dry  at  40°C. 

•  Teflon  Liners  (avoid  contact  with  fingers) 

(1)  Wash  with  detergent. 

(2)  Rinse  with  distilled  water. 

(3)  Rinse  with  acetone. 

(4)  Rinse  with  hexane  (Nanograde  or  equivalent) . 

(5)  Air  dry. 

(6)  Place  liners  in  cleaned  caps. 

(7)  Heat  to  40°C  for  2  hours. 

(8)  Allow  to  cool. 

(9)  Use  to  cap  cleaned  bottles. 
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Appendix  D 


ORGANIC  SAMPLE  COLLECTION  REQUIREMENTS 
AND 

REQUIRED  CONTAINERS.  PRESERVATION  TECHNIQUES,  AND  HOLDING  TIMES 


(USER’S  GUIDE  TO  THE  CONTRACT  LABORATORY  PROGRAM,  OCTOBER.  1984) 

AND 

(CONTRACT  LABORATORY  PROGRAM  STATEMENT  OF  WORK 
FOR  INORGANIC  ANALYSIS,  OCTOBER,  1906  REV.) 


% 


a*» 
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ALL  MEDIUM  LEVEL  SAMPLES  TO  BE  SEALED 
IN  METAL  PAINT  CAN  FOR  SHIPMENT 


:TALS  AND  CYANIDE  (CN“)  6  OZ.  p  IX  8-OZ.  WI DE-MOUTH 

ANALYSIS  j  GLASS  JAR 

[LOW  OR  MEDIUM  LEVEL*)  U 


•■ v‘»’. •<**’V*tyVW«'*V'V  <"> 


following  page  for  notes 


Polyethylene  (P)  or  Glass  (G). 


Sample  preservation  should  be  performed  immediately  upon  sample  collection. 
For  composite  samples  each  aliquot  should  be  preserved  at  the  time  of  col¬ 
lection.  When  use  of  an  automated  sampler  makes  it  impossible  to  preserve 
each  aliquot,  then  samples  may  be  preserved  by  maintaining  at  4°C  (+5*C) 
until  compositing  the  sample  splitting  is  completed. 

When  any  sample  is  to  be  shipped  by  common  carrier  or  sent  through  the 
United  States  Mails,  it  must  comply  with  the  Department  of  Transportation 
Hazardous  Materials  Regulations  (49  CFR  Part  172).  The  person  offering 
such  material  for  transportation  is  responsible  for  ensuring  such  com¬ 
pliance.  For  the  preservation  requirements  of  Table  XI,  the  Office  of 
Hazardous  Materials,  Materials  Transportation  Bureau,  Department  of 
Transportation  has  determined  that  the  Hazardous  Materials  Regulations 
do  not  apply  to  the  following  materials:  Hydrochloric  acid  (HCL)  in  water 
solutions  at  concentrations  of  0.04%  by  weight  or  less  (pH  about  1.96  or 
greater);  Nitric  acid  (HN03)  in  water  solutions  at  concentrations  of  0.15% 
by  weight  or  less  (pH  about  1.62  or  greater);  Sulfuric  acid  (H2S04)  in 
water  solutions  at  concentrations  of  0.35%  by  weight  or  less  (pH  about 
1.15  or  greater;  and  Sodium  hydroxide  (NaOH)  in  water  solution  at  con¬ 
centration  of  0.080%  by  weight  or  less  (pH  about  12.30  or  less). 

Samples  should  be  analyzed  as  soon  as  possible  after  collection.  The 
times  listed  are  the  maximum  times  that  samples  may  be  held  before 
analysis  and  still  considered  valid. 

Should  only  be  u£ed  in  the  presence  of  residual  chlorine. 

Maximum  recommended  holding  time  is  less  when  sulfide  is  present. 
Optionally,  all  samples  may  be  tested  with  lead  acetate  paper  before  the 
pH  adjustment  in  order  to  determine  if  sulfide  is  present.  If  sulfide  is 
present,  it  can  be  removed  by  the  addition  of  cadmium  nitrate  powder  until 
a  negative  spot  test  is  obtained.  The  sample  is  filtered  and  then  NaOH  is 
added  to  pH  12. 

Samples  should  be  filtered  immediately  on-site  before  adding  preservative 
for  dissolved  metals. 


DEVELOPMENT  OF  THE  QUALITY  ASSURANCE  TASK  FORCE 


Experts  from  many  scientific  and  regulatory  agencies  met  in  Las 
Vegas,  Nevada  on  February  18-20,  1987  to  discuss  Quality  Assurance/ 
Quality  Control.  The  objectives  of  the  group  were  to  exchange  ideas, 
share  resources  and  technology,  explore  creative  approaches,  address 
key  issues,  and  hopefully  develop  a  unified  plan  for  assuring  quality 
data.  A  list  of  the  attendees  is  included  in  Appendix  A.  The 
participants  were  divided  into  seven  working  groups  so  that  the 
following  topics  could  be  discussed  in  greater  detail: 


Group  1 
Group  2 
Group  3 
Group  4 
Group  5 
Group  6 
Group  7 


Quality  Assurance  Management  and  Data  Quality  Objectives 
On-Site  Auditing,  Data  Review,  and  Evaluation 
Performance  Evaluation  and  Reference  Material 
Method  Validation  and  Equivalency 

Sample  Management,  Holding  Times  and  Chain  of  Custody 
Statistics  and  Chemometrics 
Documentation  and  Data  Communication 


In  order  to  facillitate  the  continuing  exchange  of  ideas  and  resources, 
a  proposal  was  submitted  to  the  group  for  the  formation  of  a  Quality 
Assurance  Task  Force.  The  Quality  Assurance  Task  Force  would  promote 
the  continual  development  of  a  unified  approach  to  QA/QC. 


A  summary  of  the  findings  of  each  of  the  working  groups  is 
presented  below: 


GROUP  1  -  QUALITY  ASSURANCE  MANAGEMENT  AND  DATA  QUALITY  OBJECTIVES 

Group  1  emphasized  that  the  Army  and  EPA  should  define  what  types 
of  data  are  needed,  note  comparisons  and  differences  in  data  packages, 
and  address  methods  to  meet  data  requirements.  Follow-up  meetings  on 
these  and  other  differences  should  be  held.  The  decision-making 
personnel  in  each  agency  should  be  identified  and  be  responsible  for 
establishing  equivalency.  The  results  should  then  be  communicated  to 
all  ten  regions. 


GROUP  2  -  ON-SITE  AUDITING,  DATA  REVIEW,  AND  EVALUATION 

Group  2  concluded  that  there  was  no  consistency  among  the 
different  agencies  concerning  precertification  and  certification. 
EPA's  CLP  program  requires  prospective  labs  to  demonstrate,  at  their 
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own  cost,  administrative  and  technical  capabilities  before  the  contract 
is  awarded.  USATHAMA  awards  contracts,  through  a  RFP  process,  on  the 
basis  of  a  written  proposal  and  the  history  of  the  laboratory. 
Therefore,  demonstration  of  technical  profiency  is  paid  for  by 
USATHAMA.  USDA  requires  a  performance  evaluation  sample  for 
accreditation.  Failure  on  the  performance  evaluation  sample 

necessitates  that  the  laboratory  must  wait  for  six  months  before 
reapplication.  The  EPA  CLP  considers  the  postaward  performance 
evaluation  samples  to  be  a  major  topic  for  the  on-site  laboratory 
evaluation,  unlike  the  USATHAMA,  the  Navy,  or  the  USDA.  The  group 
agreed  that  the  on-site  evaluation  checklist  was  fairly  uniform,  but 
that  the  frequency  of  the  audits  and  the  level  of  corrective  action 
applied  varied  with  the  agency.  The  EPA  CLP  was  the  only  program  which 
looked  for  serious  problems  by  reconstructing  final  results  from  the 
original  raw  data  during  the  audit.  Some  members  of  the  group 
expressed  interest  in  on-site  auditing  of  the  field  sampling  process. 
Also,  the  group  proposed  that  the  issues  and  benefits  of  agencies 
sharing  laboratory  performance  information  should  be  addressed. 

Group  2  found  that  even  though  the  time  frame  allowed  for  the 
review  of  data  ranged  from  one  week  to  three  months,  the  procedures 
were  generally  the  same.  All  reviewers  looked  for  outlying  data, 
controls,  suspicious  calibrations,  etc.  The  group  did  emphasize  that 
they  did  not  feel  that  data  was  over- reviewed.  Even  though  data  was 
reviewed  in  the  same  manner,  the  application  of  the  data  depended  on 
the  end  user  and  could  vary  widely.  Differences  in  reviews  depended  on 
the  auditor's  function  in  the  overall  project  scheme.  Any  data  that 
was  involved  in  litigation  and  chain  of  custody  would  take  longer  to 
review.  The  group  decided  that  the  audit  could  be  facillitated  by 
computerized  data  scans,  organized  standard  data  packages,  and  easier 
access  to  lab  personnel  and  sampling  information. 

GROUP  3  -  PERFORMANCE  EVALUATION  AND  REFERENCE  MATERIAL 

Group  3  began  their  working  session  by  identifying  several  issues 
to  discuss.  The  group  felt  that  the  level  of  effort  required  to 
"certify  a  material"  needed  to  be  defined  and  that  increased 
traceability  of  materials  to  NBS  would  be  desirable.  Purity 
documentation  of  standard  materials  should  include  one  identity  and  two 
purity  analyses  and  only  designated  labs  should  be  allowed  to  certify 
materials.  Also  reference  materials  should  have  a  certificate  of 
analysis  with  dates  of  preparation  and  expiration,  methods  used,  and 
any  other  pertinent  data.  Interagency  cooperation  in  making  and  using 
batches  of  reference  materials  would  increase  the  amount  of  funding 
available  to  make  and  certify  Standard  Reference  Materials  from  NBS. 
The  group  felt  that  an  interagency  work  statement  was  needed  to  plan  a 
feasibility  study  on  soil  reference  material  preparation  and  analysis. 


Group  3  was  then  divided  into  three  subgroups:  definitions, 

performance  evaluation  materials,  and  reference  materials.  The 
definitions  subgroup  defined  reference  materials  as  a  general  term  for 
characterized  substances  used  for  the  following  three  functions: 
instrument  calibration,  intralaboratory  QC ,  and  interlaboratory 
performance  evaluation. 

1.  The  purpose  of  calibration  is  to  establish  the  relationship 
between  instrument  response  and  concentration.  Calibration 
is  accomplished  by  using  calibration  standards  which  are  well 
characterized  as  to  purity,  stability,  and  concentration. 

2.  The  purpose  of  intralaboratorv  quality  control  is  to  provide 

an  assessment  of  data  quality  within  a  laboratory.  This 
information  is  developed  in  part  by  daily  analysis  of  a 
laboratory  check  sample.  These  check  samples  can  be  prepared 
by  the  laboratory  or  obtained  from  an  external  source.  In 
addition,  the  laboratory  should  periodically  analyze 

externally  supplied  check  materials  of  known  composition, 
such  as  EPA's  QC  samples  and  the  SRM’s  from  NBS . 

3.  The  purpose  of  performance  evaluation  is  to  provide  an 
assessment  of  the  comparability  of  analytical  results  between 
laboratories.  Performance  evaluation  materials  (PEM's)  are 
periodically  supplied  to  the  laboratory  as  unknowns  by  the 
sponsoring  agency. 

PEM's  may  be  derived  from  natural  matrix  materials  or 
synthetically  prepared.  The  evaluation  of  laboratories  using 
PEM's  may  be  based  on  comparison  with  known  values  or  by 
comparison  of  individual  results  against  the  average 

performance . 

The  subgroup  on  performance  evaluation  materials  discussed  the 

need  for  solid  organic  performance  evaluation  samples,  acknowledging 
that  nonvolatile  evaluation  samples  are  relatively  easy  to  prepare,  but 
that  studies  were  needed  for  preparation  of  volatiles  in  solids.  A 

need  for  performance  evaluation  samples  for  explosives  in  soil  was 

expressed.  The  subgroup  decided  that  natural  contaminated  matrices 
were  preferable  to  spiked  matrices  if  possible.  The  frequency  and 
character  of  the  performance  evaluation  samples  was  discussed,  as  well 
as  the  selection  of  analytes,  matrices,  and  concentrations.  The 
necessity  of  keeping  evaluation  samples  as  blind  as  possible  was 

recognized,  with  the  minimum  frequency  being  one  blind  performance 
evaluation  sample  per  lot  of  samples.  However,  studies  should  be  made 
of  laboratory  operations  to  determine  the  optimum  frequency,  with  a 
combination  of  control  charts  and  performance  evaluation  samples  being 
the  best  approach.  The  subgroup  decided  that  the  performance 
evaluation  samples  should  be  at  multiple  levels  of  concentration 


(taking  into  account  the  method),  and  should  contain  well  characterized 
analytes  of  low  intrinsic  variability.  Both  easy  and  difficult 
analvtes  should  be  included,  as  well  as  those  that  are  easily  confused. 
Performance  evaluation  results  should  be  purged  of  proprietary 
information  and  shared  between  the  different  agencies.  Matrices  needed 
to  be  representative  of  the  sites  under  study.  Selection  of  six  to  ten 
representative  types  of  soil  was  suggested.  The  need  for  a  catalog  of 
sources  of  environmental  performance  evaluation  samples  was  expressed. 
The  subgroup  also  discussed  the  evaluation  of  the  performance  of 
laboratories  from  evaluation  sample  data.  The  use  of  the  results  and 
the  formation  of  data  quality  objectives  should  be  consistent  with  the 
end  use  of  the  data  such  as  risk  assessment  and  legal  considerations. 
Education  of  the  end  users  of  the  data  was  suggested  because  of 
misconceptions  concerning  the  width  of  windows  for  evaluating  the 
results.  Participants  also  reaffirmed  that  performance  evaluation 
sample  results  only  comprised  one  portion  of  the  laboratory  evaluation 
and  that  results  would  be  of  marginal  value  if  criteria  limits  were  set 
near  method  detection  limits.  Comments  were  made  that  dilution  errors 
were  a  problem  with  high  level  performance  evaluation  samples  and 
outliers  on  the.  high  and  low  sides  needed  to  be  handled  differently. 


.-ROl'P  a  -  METHOD  VALIDATION  AND  EQUIVALENCY 

The  first  concern  of  Group  4  was  to  define  method  validation  in 

•  h e  t  o  1  1  ow i  r.g  marine r  : 

Me  •  hod  I  i  da  t  i  on  is  a  process  starting  with  definition  of 

method  analytes  and  sample  matrices  followed  by 

identification  of  suitable  existing  methods  for  conducting 
t  he  anal vs  is .  A  method  is  selected,  optimized  and  validated 
in  i  sir.-,!.-  laboratory  study  which  must  include  determination 
•f  noth'-d  det  «-e{  ion  limit,  precision  and  bias  in  a  range  of 
■  '■*  in:  erest  and  also  include  ruggedness  testing  and 

■■■  :  t  mplete  met  nod  protocol.  Following  a  rigorous 

i  ng  1  »•  1  s:  udv  .  the  method  undergoes  collaborative  testing 
mini  •.urn  >>f  six  to  eight  laboratories.  The  method  is 
•  'f.s ;  •«,  :  ,'d  -  .  he  validated  if  the  written  protocol  can  be 
t  . '.  w-  i  hv  the  participating  laboratories,  the  method  is 
cii  foe  *  he  tested  matrices,  and  if  the  precision  and 

euru<v  of  the  collaborative  results  are  within  the  limits 

■■r  *  in  t  he  Data  Quality  Objectives. 

The  group  also  recognized  that  method  validation  is  a  separate 
ess  from  • ertif ication  of  a  laboratory  to  perform  a  method.  In 
i. dating  a  method,  the  natural  environmental  matrix  was  preferred  if 
v.  process  of  locating  and  characterizing  the  matrix  did  not  exhaust 
he  ivuiluble  resources.  Fortified  natural  matrices  could  be  used  as 
w*  11  as  a  totally  synthetic  matrix.  Problems  associated  with  obtaining 
laboratories  to  participate  in  interlaboratory  studies  was  discussed. 
The  group  felt  that  competition  among  the  laboratories  might  make  them 
commit  to  a  validation  effort,  but  there  were  no  guarantees  that  the 
data  would  he  delivered  in  a  timely  manner. 


Guidelines  for  conducting  dynamic  validation  were  developed  by  the 
group.  It  should  only  be  used  when  more  than  20  laboratories  are 
participating,  the  method  must  be  based  on  a  previously  tested  method 
for  which  there  is  a  high  degree  of  confidence  that  its  performance 
will  meet  or  exceed  program  requirements,  and  the  method  must  include  a 
strong  quality  control  program. 

Group  4  recommended  that  the  EPA  Superfund  staff  adopt  a  policy 
which  would  allow  other  federal  agencies  to  demonstrate  equivalency  of 
their  methods  to  Superfund  methods.  Problems  mentioned  which  could 
result  from  adopting  such  a  policy  were  different  reporting 
requirements  and  proliferation  of  equivalent  methods  which  would  make 
data  review  more  difficult. 

The  group  recommended  that  other  federal  agencies  be  invited  to 
attend  future  caucuses  and  that  the  QA  workshop  should  be  reconvened  in 
one  year  or  less . 

GROUP  5  -  SAMPLE  MANAGEMENT,  HOLDING  TIMES  AND  CHAIN  OF  CUSTODY 

Group  5  recognized  that  a  uniform  sample  definition  would  be 
desirable,  but  may  not  be  possible.  However,  sample  terms  must  be 
defined  and  mutually  understood  by  all  agencies  involved.  QA 
procedures  for  field  sampling,  field  logs,  and  chain-of-custody 
documentation  should  be  uniform.  Total  compliance  in  maintaining 
chain-of-custody  could  be  very  difficult  with  more  automated  analyses. 
More  stringent  QA  concerning  field  sampling  is  needed  since  this 
represents  a  huge  source  of  error.  Some  estimate  of  this  error  would 
be  desirable.  The  group  felt  that  information  on  the  validity  of 

holding  times  would  be  desirable  in  unifying  specifications  among  the 
various  agencies.  Requirements  for  reanalysis  were  different  among  the 
agencies  if  checking  of  data  revealed  that  analyses  were 

inappropriately  performed.  A  centralized  database  as  a  means  of 
obtaining  summary  information  on  laboratories  such  as  current 
standing,  date  of  most  recent  audit,  and  date  of  most  recent 

performance  evaluation  sample  analysis  would  be  advantageous  to  all 
agencies  concerned. 

GROUP  6  -  STATISTICS  AND  CHEMOMETRICS 

Group  6  raised  five  issues  for  discussion.  The  first  issue  was 
detection  limits.  The  group  established  that  the  detection  limit 

represents  a  concentration  where  decisions  about  presence  or  absence 
are  made  and  that  the  quantitation  (reporting)  limit  is  at  some 

concentration  above  the  detection  limit.  The  detection  limit  is  highly 
dependent  on  the  individual  characteristics  of  the  apparatus,  analyst, 
method,  etc.  Data  was  shown  to  suggest  that  EPA's  MDL  and  THAMA's 
Hubaux  and  Vos  estimates  from  some  data  sets  show  a  maximum  difference 


a 


in  ratio  of  1.5.  The  group  concluded  that  the  two  procedures  were  not 
as  different  as  thought  at  first,  and  expressed  a  need  for  more 
information  on  how  to  set  limits  for  multi-analyte  methods,  using 
surrogates  to  evaluate  matrix  variations,  more  comparative  evaluations 
of  the  different  methods  of  estimating  detection  limits,  more  education 
on  the  variables  which  are  included  in  the  detection  limit  estimates, 
and  a  determination  of  the  most  effective  way  to  specify  concentration 
limits  and  evaluate  inherent  risks. 


The  second  issue  raised  was  chemometr ics .  During  this  discussion, 
the  following  needs  were  expressed:  investigation  of  applications  of 
composite  sampling  techniques  to  environmental  monitoring,  estimation 
of  variablility  due  to  sampling,  improved  laboratory  subsampling 
protocols,  and  more  nested  experimental  programs  to  provide  objective 
estimates  based  on  real  samples  of  the  following:  field  sampling 
variability,  lab  subsampling/preparation,  and  analytical  variablility. 


u 
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The  third  issue  that  the  group  discussed  was  the  development  of 
performance  evaluation  sample  criteria.  Double -blind  performance 
evaluation  sample  submission  was  considered  ideal  when  feasible  at  a 
frequency  consistent  with  the  needs  of  the  program  and  cost  benefit 
goals.  Ultimately  the  group  wanted  to  see  capability  limits 
established  for  the  performance  evaluation  standards  for  various 
methods  and  for  different  types  of  evaluation  materials.  In 
establishing  these  criteria,  the  problem  of  editing  data  to  exclude 
true  "outliers"  without  unduly  truncating  data  sets  was  recognized. 
The  question  was  also  raised  about  the  effect  of  a  large  number  of 
outliers  in  a  data  set  upon  future  repeatability.  A  suggestion  was 
made  to  use  the  Biweight  Robust  Estimation  Procedure  (JASA,  June  1982) 
which  provides  the  basis  for  USEPA  performance  evaluation  criteria 
limits  for  water  analysis.  Some  out-of -control  data  could  be  discarded 
if  the  frequency  of  performance  evaluation  checks  is  increased. 

The  group  labeled  the  fourth  issue  as  desien/analvsis 
comparability .  Essentially,  improved  communication  between  lab 
personnel  and  field  samplers  was  a  necessity  in  order  to  carefully 
formulate  all  sampling  protocols  and  analysis  in  advance  so  that  all 
statistical  computations  would  be  compatible  with  the  actual 
performance  of  the  experiments.  Any  uncontrolled  variables  in  the 
procedures  also  needed  to  be  noted  to  aid  in  the  design  process  as  well 
as  final  use  of  the  data  (qualitative  vs.  quantitative). 


Concerning  the  last  issue,  validation  of  reference  materials,  the 
group  required  detailed  procedures  to  establish  homogeneity  and 
stability . 


GROUP  7  -  DOCUMENTATION  AND  DATA  COMMUNICATION 


Group  7  agreed  that  the  quality  and  quantity  of  the  documentation 
required  for  a  program  varies  depending  on  the  end  use  of  the  data  f rom 
the  most  intensive  documentation  necessary  for  litigation  purposes  to 
minimum  documentation  only  needed  for  character izat ion/informa t i on 
gathering  to  make  rapid  decisions.  The  group  stated  that  sampling  and 
analytical  contracts  should  require  the  level  of  documentation 
necessary  for  the  program's  purposes  with  the  long  term  goal  that  the 
various  agencies  would  reach  some  consensus  on  what  documentation 
should  be  required.  Meeting  this  goal  would  solve  such  problems  as 
agreement  of  EPA  and  DOD  on  documentation  when  both  agencies  are 
involved  in  sites  on  DOD  facilities  and  alleviate  the  frustration  of 
contractor  labs  who  are  required  to  follow  different  documentation  and 
reporting  requirements  depending  on  the  agency.  The  group  questioned 
whether  software  systems  could  be  developed  and  implemented  to  process 
some  of  the  quality  control  elements  common  to  all  the  agencies  and 
provide  some  documentation.  USATHAMA  followed  up  this  discussion  with 
a  presentation  of  their  computer  software. 
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